Sample-Dependent Adaptive Temperature Scaling for Improved Calibration

It is now well known that neural networks can be wrong with high confidence in their predictions, leading to poor calibration. The most common post-hoc approach compensate for this perform temperature scaling, which adjusts the confidences of predictions on any input by scaling logits a fixed value. Whilst typically improves average calibration across whole test dataset, improvement reduces individual irrespective whether classification given correct or incorrect. With insight, we base our method observation different samples contribute error varying amounts, some needing increase and others decrease it. Therefore, each input, propose predict value, allowing us adjust mismatch between accuracy at finer granularity. Our applied post-hoc, enabling it very fast negligible memory footprint off-the-shelf pre-trained classifiers. We ResNet50 WideResNet28-10 architectures using CIFAR10/100 Tiny-ImageNet datasets, showing producing per-data-point temperatures expected set.